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7 VARIABLE PERSPECTIVE VIEW OF VIDEO IMAGES 
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10 TECHNICAL FIELD 

1 1 Certain embodiment of this invention relate generally to the field of video 

1 2 display. More particularly, in certain embodiments, this invention relates to display 

13 of a variable perspective video image by use of a television's picture-in-picture 

14 feature and multiple video streams. 
15 

16 BACKGROUND 

1 7 The DVD (Digital Versatile Disc) video format provides for multiple viewing 

18 angles. This is accomplished by providing multiple streams of video taken from 

19 multiple cameras. The idea is for the multiple cameras to take multiple views of 

20 the same scene that the user may select from. Using this video format, the viewer 

21 with an appropriately equipped playback device can select the view that is most 

22 appealing. While this feature is available, heretofore, it has been sparsely utilized. 

23 Moreover, the available perspectives are from several distinct camera angles that 

24 are discretely selected by the user to provide an abrupt change in perspective. 
25 

26 OVERVIEW OF CERTAIN EMBODIMENTS 

27 The present invention relates, in certain embodiments, generally to display 

28 of a selective view of a scene using a television's picture-in-picture feature. 

29 Objects, advantages and features of the invention will become apparent to those 
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1 skilled in the art upon consideration of the following detailed description of the 

2 invention. 

3 A method of displaying a view of a scene on an electronic display consistent 

4 with certain embodiments involves presenting a main window and a secondary 

5 window adjacent the main window. A first and a second image are provided, 

6 wherein the first and second images overlap one another by at least 50%. A 

7 portion of the first image is removed and a remainder of the first image is displayed 

8 in the main window. A portion of the second image is removed and a remainder 

9 of the second image is displayed in the secondary window. In this manner, a 

10 composite image made up of the remainder of the first image displayed adjacent 

1 1 the remainder of the second image provides a selected view extracted from a total 

12 scene captured in the sum of the first and second images. 

13 A device for producing a view of a scene consistent with certain 

1 4 embodiments of the invention has a demultiplexer that receives an input stream as 

1 5 an input and produces a first video stream and a second video stream as outputs, 

16 wherein the first video stream represents a first video image of the scene and 

17 wherein the second video stream represents a second video image of the scene. 

1 8 A main decoder receives the first video stream and a secondary decoder receives 

1 9 the secondary video stream. Portions of the first and second images are removed 

20 to leave remaining portions of the first and second images. An image combiner 

21 combines the first and second images to produce a composite image, wherein the 

22 composite image represent a view of the scene. 

23 A method of creating multiple images for facilitating display of a selected 

24 view of a scene consistent with certain embodiments involves capturing a first 

25 image of a scene from a location using a first camera angle; capturing a second 

26 image of the scene from the location using a second camera angle, wherein the 

27 first and second images have at least 50% overlap; associating the first image with 

28 a first packet identifier; associating the second image with a second packet 

29 identifier; and formatting the first and second images in a digital format. 
30 
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1 Another method of displaying an image on an electronic display consistent 

2 with certain embodiments of the invention involves presenting a main window; 

3 presenting a secondary window adjacent the main window; providing a first and a 

4 second image, wherein the first and second images overlap one another; stitching 

5 together the first and second images to produce a panoramic image; and from the 

6 panoramic image, generating first and second display images for display in the 

7 main and secondary windows such that a view from the panoramic image spans 

8 the main and secondary windows. 

9 Another method of displaying a view of a scene on an electronic display 

10 consistent with certain embodiments involves presenting a main window; 

1 1 presenting a secondary window adjacent the main window; providing a first and a 

12 second image, wherein the first and second images overlap one another by J%; 

13 removing a portion of the first image and displaying a remainder of the first image 

14 in the main window; removing a portion of the second image and displaying a 

15 remainder of the second image in the secondary window; and wherein, a 

16 composite image comprising the remainder of the first image displayed adjacent 

1 7 the remainder of the second image provides a selected view extracted from a total 

18 scene captured in the sum of the first and second images. 

1 9 The above overviews are intended only to illustrate exemplary embodiments 

20 of the invention, which will be best understood in conjunction with the detailed 

21 description to follow, and are not intended to limit the scope of the appended 

22 claims. 
23 

24 BRIEF DESCRIPTION OF THE DRAWINGS 

25 The features of the invention believed to be novel are set forth with 

26 particularity in the appended claims. The invention itself however, both as to 

27 organization and method of operation, together with objects and advantages 

28 thereof, may be best understood by reference to the following detailed description 

29 of the invention, which describes certain exemplary embodiments of the invention, 

30 taken in conjunction with the accompanying drawings in which: 

Docket No.: SNY-T5574 -3- PATENT 



1 FIGURE 1, which is made up of FIGURE 1a, 1b and 1c, illustrates multiple 

2 image capture by multiple cameras in a manner consistent with certain 

3 embodiments of the present invention. 

4 FIGURE 2 is a composite image made up of the three overlapping images 

5 captured in FIGURE 1 in a manner consistent with certain embodiments of the 

6 present invention. 

7 FIGURE 3 is a flow chart of an image capture process consistent with 

8 certain embodiments of the present invention. 

9 FIGURE 4 is a flow chart of an image presentation process consistent with 

1 0 certain embodiments of the present invention. 

1 1 FIGURE 5, which is made up of FIGURES 5a - 5f, depicts panning to the 

12 right in a manner consistent with certain embodiments of the present invention. 

1 3 FIGURE 6 is a block diagram of an exemplary receiver or playback device 

14 suitable for presenting a panned view to a display in a manner consistent with 

1 5 certain embodiments of the present invention. 

16 FIGURE 7 is a flow chart depicting a process for panning right in a manner 

1 7 consistent with certain embodiments of the present invention. 

18 FIGURE 8 is a flow chart depicting a process for panning left in a manner 

19 consistent with certain embodiments of the present invention. 

20 FIGURE 9 is a flow chart of an image capture process for an alternative 

21 embodiment consistent with the present invention. 

22 FIGURE 10 is a flow chart of an image presentation process consistent with 

23 certain embodiments of the present invention. 
24 

25 DETAILED DESCRIPTION 

26 While this invention is susceptible of embodiment in many different forms, 

27 there is shown in the drawings and will herein be described in detail specific 

28 embodiments, with the understanding that the present disclosure is to be 

29 considered as an example of the principles of the invention and not intended to limit 
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1 the invention to the specific embodiments shown and described. In the description 

2 below, like reference numerals are used to describe the same, similar or 

3 corresponding parts in the several views of the drawings. 

4 For purposes of this document, the term "image" is intended to mean an 

5 image captured by a camera or other recording device and the various data 

6 streams that can be used to represent such an image. The term "view" is used to 

7 describe the representation of an image or a combination of images presented to 

8 a viewer. The term "scene" is used to mean a sum of all images captured from 

9 multiple camera angles. 

10 The present invention, in certain embodiments thereof, provide a 

1 1 mechanism for permitting a viewer to view an apparently continuously variable 

12 perspective of an image by panning across the scene. This process is made 

13 possible, in certain embodiments by starting with multiple perspectives being 

1 4 captured by a video tape recorder or film camera (either still orfull motion). Turning 

15 now to FIGURE 1, made up of FIGURE 1a, 1b and 1c, the process begins with 

16 capturing two or more (three are illustrated, but this should not be considered 

17 limiting) images of a scene. In FIGURE 1a, the left side of a city-scape scene is 

18 captured as an image by camera 10a. In FIGURE 1b, the center of a city-scape 

19 scene is captured as an image by camera 10b. In FIGURE 1c, the left side of a 

20 city-scape scene is captured as an image by camera 10c. 

21 Cameras 10a, 10b and 10c may be integrated into a single camera device 

22 or separate devices may be used. But in any event, the cameras should capture 

23 the images from the same location with different viewing angles. However, as long 

24 as the images can be made to overlap as described, any process for creating the 

25 multiple overlapping images is acceptable within the present invention. Such a 

26 camera device may incorporate any number 2 through N cameras. Any number of 

27 cameras and camera angles can be provided and can even be arranged to provide 

28 a full 360 degrees by providing enough camera angles such that a pan can be 

29 carried out in a full circle. Moreover, although this illustrative embodiment only 
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1 shows three camera angles with the cameras angles capturing 50% overlap in the 

2 horizontal direction, vertically overlapping camera angles can also be used to 

3 facilitate panning up or down or in any direction when multiple camera angles are 

4 provided with both horizontal and vertical coverage. In this preferred embodiment, 

5 the cameras capture images that overlap the adjacent images by at least 50%, but 

6 in other embodiments, minimal overlap is required, as will be discussed later. 

7 These images can then be stored and digitally transmitted as described later. 

8 Thus, by reference to FIGURE 2 it can be seen that three separate images 

9 14, 16 and 18 with 50% overlap are obtained from cameras 10a, 10b and 10c 

10 respectively to represent the exemplary city-scape scene. By overlaying these 

11 images with the overlaps aligned as shown, the image makes up a wider 

12 perspective of the scene than any single camera captures. Using exactly 50% 

13 overlay, three camera images can create a superimposed image that is twice the 

14 width of a single camera's image. This superimposed image represents all 

1 5 available views of the scene for this example. 

16 The 50% overlap provides for the ability to have fixed size windows for the 

17 main and secondary (PIP) windows in order to provide the desired ability to pan. 

18 However, one skilled in the art will appreciate that by also providing for variability 

19 of the window sizes, a smaller amount of overlap can be used to still achieve the 

20 panning effect. This is accomplished by adjusting the size of the view displayed 

21 in each window (one expands while the other contracts) in order to simulate the 

22 pan. When a limit on an image is reached, the window sizes are again changed 

23 and new set of images are used to create the next panned view. 

24 The process of capturing and utilizing these images, is described in the 

25 process of the flow chart of FIGURE 3. This flow chart summarizes the process 

26 described above starting at 22. At 26, the N images of a particular scene are 

27 captured from N cameras (or the equivalent) with each image overlapping adjacent 

28 images by at least 50%. In accordance with this embodiment, the N different 

29 images can be formatted as an MPEG (Moving Pictures Expert Group) format or 
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1 other suitable digital format. In so doing, each of the N images and associated 

2 data streams can be assigned a different packet identifier (PID) or set of packet 

3 identifiers (or equivalent packet identification mechanism) at 30 in order to 

4 associate each packet with the data stream or file of a particular image. Once the 

5 images are so formatted, they can be stored and / or transmitted to a receiver at 34. 

6 This process ends at 38. 

7 Once these images are stored on an electronic storage medium or 

8 transmitted to the receiver, a panning operation can be carried out by the receiver 

9 or a media player under user control as described in one embodiment by the flow 

1 0 chart of FIGURE 4 starting at 44. The images, identified by distinct PIDs for each 

11 image data stream, are received or retrieved from storage, or downloaded or 

12 streamed at 48. A pair of windows, e.g., in the case of a television display, a main 

1 3 window and a picture-in-picture (PIP) window, are displayed adjacent one another 

14 at 52. For simplicity of explanation, it will be assumed that the main window is 

15 always to the left and the PIP window is always to the right. The windows can 

16 occupy the left and right halves of the display screen if desired and are one half the 

17 width of a normal display. A user selected (or initially a default) view 56 of the 

18 images is displayed in the two side by side windows to represent a single view. 

19 In order to display the selected view, the overlapping images and portions 

20 of overlapping images are identified at 60 to produce the selected view. Then, for 

21 each frame of the video image at 64, the main and secondary views are 

22 constructed at 68 by slicing selected portions of the selected images to remove the 

23 unused portions. One of the sliced images is displayed on the main window while 

24 the other is displayed on the secondary (e.g., PIP) window at 72. Since the 

25 windows are positioned side by side, the two half images are displayed to produce 

26 the whole selected view of the scene to the viewer. If the last frame has not been 

27 reached at 76 and a pan command has not been received at 80, the process 

28 proceeds as described for each frame in the data streams. Once the last frame is 

29 received, the process ends at 84. If a pan command is issued by the user to either 

30 pan left or right (or up or down or in any other direction in other embodiments), 
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1 control returns to 60 where the process again identifies the images needed to 

2 produce the selected view. 

3 As will become clear later, by use of the present process, very little 

4 computing power is needed to generate a panning effect as described. The pan 

5 command received (e.g., by a left or right arrow control on a remote controller), the 

6 images are selected and sliced according to the degree of left or right pan 

7 requested. Since each data stream representing each image is easily identified by 

8 the PID or PIDs associated therewith, the receiver can easily divert one stream to 

9 a main decoder and a secondary stream to a secondary decoder (e.g., a PIP 

1 0 decoder). The decoders can further be instructed to slice the image vertically (or 

1 1 horizontally) in an appropriate location and the respective images displayed on the 

1 2 main and secondary windows of the display. 

1 3 The process of FIGURE 4 above is illustrated in FIGURES 5a - 5f. Assume 

1 4 for purposes of this illustration, that a full image can be represented by six vertical 

15 columns of pixels (or sets of pixels). Clearly, most images will require far more 

16 columns of pixels to provide a meaningful display, but, for ease of explanation, 

1 7 consider that only six are required. Consistent with a 50% overlap in the images, 

1 8 a first image 1 00 contains pixel columns A through F, second image 1 02 contains 

19 pixel columns D through I and third image 104 contains pixel columns G through 

20 L. This provides enough redundant information to permit assembly of any desired 

21 view of the scene using two of the video data streams containing adjacent 

22 overlapping images. To display a leftmost view of the scene as shown in FIGURE 

23 5a, columns A, B and C can be extracted from image 100 and displayed on the 

24 main window 108, while columns D, E and F extracted from image 102 and 

25 displayed on the PIP or other secondary window 1 1 0. (Alternatively, all six columns 

26 of pixels can be taken from image 1 00.) 

27 If a command is received to pan to the right by one pixel column, the image 

28 is constructed as shown in FIGURE 5b. To display this view, columns B, C and D 

29 can be extracted from image 100 and displayed on the main window 108, while 
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1 columns E, F and G extracted from image 102 and displayed on the PIP or other 

2 secondary window 1 1 0. 

3 If a command is received to again pan to the right by one pixel column, the 

4 image is constructed as shown in FIGURE 5c. To display this view, columns C, D 

5 and E can be extracted from image 100 and displayed on the main window 108, 

6 while columns F, G and H are extracted from image 102 and displayed on the PIP 

7 or other secondary window 1 1 0. 

8 If another command is received to pan to the right by one pixel column, the 

9 image is constructed as shown in FIGURE 5d. To display this view, columns D, 

10 E and F can be extracted from image 1 00 or image 1 02 and displayed on the main 

1 1 window 108, while columns G, H and I can be extracted from image 102 or 104 and 

1 2 displayed on the PIP or other secondary window 1 1 0. 

13 If a command is again received to pan to the right by one pixel column, the 

14 image is constructed as shown in FIGURE 5e. To display this view, columns E, F 

15 and G can be extracted from image 102 and displayed on the main window 108, 

16 while columns H, I and J extracted from image 104 and displayed on the PIP or 

1 7 other secondary window 1 1 0. 

18 Finally, for purposes of this example, if another command is received to pan 

1 9 to the right by one pixel column, the image is constructed as shown in FIGURE 5f. 

20 To display this view, columns F, G and H can be extracted from image 102 and 

21 displayed on the main window 108, while columns I, J and K extracted from image 

22 1 04 and displayed on the PIP or other secondary window 1 1 0. 

23 While the example of FIGURE 5 depicts only right panning, those skilled in 

24 the art will readily understand, upon consideration of the present teaching, the 

25 operation of a left pan (or an up or down pan). A left pan scenario can be 

26 visualized by starting with FIGURE 5f and working backwards toward FIGURE 5a. 

27 A receiver (e.g., a television set top box, or television) or playback system 

28 (e.g., a DVD player or personal computer systetm) suitable for presenting such a 

29 panning view to a suitable display is depicted in block diagram form in FIGURE 6. 



Docket No.: SNY-T5574 



PATENT 



1 In this exemplary system, a transport stream containing possibly many video and 

2 associated data streams is provided to a demultiplexer 150 serving as a PID filter 

3 that selects a stream of video data based upon the PID as instructed by a 

4 controller, e.g., a microcomputer, 154. Controller 154 operates under a user's 

5 control via a user interface 158 wherein the user can provide instructions to the 

6 system to pan left or right (or up or down, etc.). Controller 1 54 provides oversight 

7 and control operations to all functional blocks as illustrated by broken lined arrows. 

8 Controller 154 instructs demultiplexer 150 which video streams (as 

9 identified by PIDs) are to be directed to a main decoder 162 and a secondary 

10 decoder 166 (e.g., a PIP decoder). In this manner, the 50% or greater overlapped 

11 images can individually each be directed to a single decoder for decoding and 

12 slicing. The slicing can be carried out in the decoders themselves under program 

13 control from the controller 154, or may be carried out in a separate slicing circuit 

14 (not shown) or using any other suitable mechanism. In this manner, no complex 

15 calculations are needed to implement the panning operation. Under instructions 

16 from controller 154, the demultiplexer 150 directs a selected stream of video to the 

17 main decoder 162 and the secondary decoder 166. The controller instructs the 

18 main decoder 162 and secondary decoder 166 to appropriately slice their 

19 respective images to create the desired view (in this embodiment). The sliced 

20 images are then combined in a combiner 172 that creates a composite image 

21 suitable for display on the display, with the main and secondary images situated 

22 adjacent one another to create the desired view. In certain other embodiments, the 

23 slicing of the individual images can be carried out in the combiner 172 under 

24 direction of the controller 154. Display interface 176 places the composite image 

25 from combiner 154 into an appropriate format (e.g., NTSC, PAL, VSGA, etc.) for 

26 display on the display device at hand. 

27 FIGURE 7 describes one exemplary process that can be used by controller 

28 154 in controlling a right pan operation starting at 200. For purposes of this 

29 process, the PID values assigned to the N video streams are considered to be 

30 numbered from left image to right image as PID 0, PID 1, ... , PID N-2, PID N-1. In 
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1 this manner, the terminology of minimum or maximum PID is associated with 

2 leftmost image or rightmost image respectively, etc. At 204, if a pan right 

3 command is received, control passes to 208, otherwise, the process awaits receipt 

4 of a pan right command. If the secondary (PIP) display is displaying the video 

5 stream with the greatest PID value and is all the way to the right, no action is taken 

6 at 208 since no further panning is possible to the right. If not at 208, and if the main 

7 display is at the right of the current image at 21 2, then the video stream for the next 

8 higher value PID is sent to the main decoder at 21 6. Next the main view is placed 

9 at the right of the new PID at 220 and control passes to 224. At 224, the main view 

10 is shifted by X (corresponding to a shift amount designated in the shift right 

1 1 command. If the main view is not at the right of the current image at 212, control 

12 passes directly to 224, bypassing 216 and 220. 

13 At 228, if the secondary display is all the way to the right of it's current 

1 4 image, the PID value is incremented at 232 to move to the next image to the right 

1 5 and the new PID valued video stream is sent to the secondary decoder. At 234 the 

16 secondary view is set to the left side of the image represented by the current PID 

1 7 value. Control then passes to 238 where the PIP view is also shifted to the right by 

1 8 x and control returns to 204 to await the next pan command. If the secondary view 

1 9 is not at the right of the current image at 228, control passes directly from 228 to 

20 238, bypassing 232 and 234. 

21 FIGURE 8 describes one exemplary process that can be used by controller 

22 1 54 in controlling a left pan operation starting at 300. At 304, if a pan left command 

23 is received, control passes to 308, otherwise, the process awaits receipt of a pan 

24 left command. If the secondary (PIP) display is displaying the video stream with 

25 the smallest PID value and is all the way to the left, no action is taken at 308 since 

26 no further panning is possible to the left. If not at 308, and if the main display is at 

27 the left of the current image at 31 2, then the video stream for the next lower value 

28 PID is sent to the main decoder at 316. Next the main view is placed at the right 

29 of the new PID at 320 and control passes to 324. At 324, the main view is shifted 

30 by X (corresponding to a shift amount designated in the shift right command) to the 
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1 left. If the main view is not at the left of the current image at 312, control passes 

2 directly to 324, bypassing 316 and 320. 

3 At 328, if the secondary display is all the way to the left of it's current image, 

4 the PID value is incremented at 332 to move to the next image to the left and the 

5 new PID valued video stream is sent to the secondary decoder. At 334 the 

6 secondary view is set to the right side of the image represented by the current PID 

7 value. Control then passes to 338 where the PIP view is also shifted to the left by 

8 x and control returns to 304 to await the next pan command. If the secondary view 

9 is not at the left of the current image at 328, control passes directly from 328 to 

10 338, bypassing 332 and 334. 

1 1 The above described process are easily implemented with relatively low 

1 2 amounts of computing power, since the video streams can be readily distinguished 

13 by their PID and directed to the appropriate decoder. The decoder or a combiner 

14 or other signal processing device can then be programmed to slice the image as 

15 desired to create the left and right halves of the particular view selected. 

16 In an alternative embodiment, a similar effect can be achieved without need 

1 7 for the 50% or more overlap in the captured images, but at the expense of possibly 

1 8 greater processing power at the receiver / decoder side. FIGURE 9 is a flow chart 

19 of an image capture process for such alternative embodiment consistent with the 

20 present invention starting at 400. This process is similar to the prior process except 

21 for the lack of constraint on the amount of overlap. At 404, N images are captured 

22 from N cameras or equivalent from N different angles, but with the cameras located 

23 at the same point. In this case, the images are only slightly overlapped to facilitate 

24 stitching together of the images. Theoretically, a continuous pan can be achieved 

25 with no overlap if the images begin and end precisely at the same line. For 

26 purposes of this document, images that begin and end at substantially the same 

27 line will also be considered to be overlapped if they can be stitched together to 

28 render a composite panoramic scene. At 408, N different PID values are assigned 

29 to the N images that are then stored or transmitted to a receiver at 412. The 

30 process ends at 416. 
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1 Once this set of images is captured using the process just described, the 

2 decoding or playback process can be carried out. FIGURE 10 is a flow chart of an 

3 image presentation process consistent with this alternative embodiment of the 

4 present invention starting at 420. The images identified by PIDs or other identifiers 

5 are received or retrieved at 424. At 428, main and secondary windows are 

6 presented side by side and adjacent one another. A view is selected by the user 

7 at 432, or initially, a default view is established. The process identifies which of the 

8 N images are needed for the selected view at 436. At 440, for each frame the 

9 images are stitched together to create what amounts to a panoramic image from 

10 two (or more) adjacent images using known image stitching technology at 444. 

1 1 This panoramic image is then divided into right and left halves at 448 and the right 

12 and left halves are sent to a decoder for display side by side in the main and 

1 3 secondary windows at 452. If the last frame has not been reached at 456, and no 

1 4 command has been received to execute a pan at 460, the process continues at 440 

1 5 with the next frame. If, however, the user executes another pan command at 460, 

16 control returns to 436 where the new images needed for the view are selected by 

17 virtue of the pan command and the process continues. When the last frame is 

1 8 received at 456, the process ends at 464. 

19 In another alternative embodiment, a similar effect can again be achieved 

20 without need for the 50% or more overlap in the captured images. FIGURE 11 is 

21 a flow chart of an image capture process for such alternative embodiment 

22 consistent with the present invention starting at 500. This process is similar to the 

23 prior image capture processes. At 504, N images are captured from N cameras 

24 or equivalent from N different angles, but with the cameras located at the same 

25 point. In this case, the images are overlapped by any selected overlap of J% (e.g., 

26 10%, 25, 40%, etc.). At 508, N different PID values are assigned to the N images 

27 that are then stored or transmitted to a receiver at 512. The process ends at 516. 

28 Again, the number of images can be any suitable number of two or more images 



Docket No.: SNY-T5574 



PATENT 



1 and may even be arranged to produce a 360 degree pan if desired, as with the 

2 other embodiments. 

3 Once this set of images is captured using the process just described, the 

4 decoding or playback process can be carried out. FIGURE 12 is a flow chart of an 

5 image presentation process consistent with this additional alternative embodiment 

6 of the present invention starting at 520. The images identified by PIDs or other 

7 identifiers are received or retrieved at 524. At 528, main and secondary windows 

8 are presented side by side and adjacent one another. However, in this 

9 embodiment, the size of the windows is dependent upon the amount of overlap and 

1 0 the location of the view. 

11 A view is selected by the user at 532, or initially, a default view is 

12 established. The process, at 536, identifies which of the N images are needed for 

13 the selected view. At 540, for each frame, portions of images are selected to 

14 create the selected view by using no more than the available J% overlap at 544. 

1 5 The window sizes are selected to display the desired view by presenting right and 

16 left portions of a size determined by the view and the available overlap at 548. The 

1 7 right and left portions of the view are sent to decoders for display side by side in the 

18 main and secondary windows at 552. If the last frame has not been reached at 

19 556, and no command has been received to execute a pan at 560, the process 

20 continues at 540 with the next frame. If, however, the user executes another pan 

21 command at 560, control returns to 536 where the new images needed for the view 

22 selected by virtue of the pan command are presented and the process continues. 

23 When the last frame is received at 556, the process ends at 564. 

24 In this embodiment, each frame of a view may be produced by not only 

25 selection of a particular segment of a pair of images for display, but also by 

26 possibly adjusting the size of the windows displaying the images. By way of 

27 example, and not limitation, assume that the image overlap (J) is 25% on adjacent 

28 images. The far left image may be displayed in a left (main) window occupying 

29 75% of the display, and in a left (secondary) window displaying 25% of the adjacent 

30 window. When a far right image is reached (again having 25% overlap with the 
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1 image to its immediate left, the image can continue to pan by changing the sizes 

2 of the two windows. The left window decreases in size while the right window 

3 increases in size until the far right is reached. At this point, the left window would 

4 occupy 25% of the view while the right window would occupy 75% of the view. 

5 While the present invention has been described in terms of exemplary 

6 embodiments in which left and right panning are described, in other embodiments, 

7 panning can also be carried out up and down or at any other angle. This is 

8 accomplished using similar algorithms to those described above on multiple 

9 images take with suitable camera angles. Moreover, it is possible to provide 

10 panning in all directions by providing enough images that have suitable overlap in 

11 both vertical and horizontal directions. Other variations will also occur to those 

12 skilled in the art upon consideration of the current teachings. 

13 Those skilled in the art will recognize, upon consideration of the present 

14 teachings, that the present invention has been described in terms of exemplary 

1 5 embodiments based upon use of a programmed processor such as controller 1 54. 

16 However, the invention should not be so limited, since the present invention could 

17 be implemented using hardware component equivalents such as special purpose 

18 hardware and/or dedicated processors which are equivalents to the invention as 

19 described and claimed. Similarly, general purpose computers, microprocessor 

20 based computers, micro-controllers, optical computers, analog computers, 

21 dedicated processors and/or dedicated hard wired logic may be used to construct 

22 alternative equivalent embodiments of the present invention. 

23 Those skilled in the art will appreciate, in view of this teaching, that the 

24 program steps and associated data used to implement the embodiments described 

25 above can be implemented using disc storage as well as other forms of storage 

26 such as for example Read Only Memory (ROM) devices, Random Access Memory 

27 (RAM) devices; optical storage elements, magnetic storage elements, magneto- 

28 optical storage elements, flash memory, core memory and/or other equivalent 

29 storage technologies without departing from the present invention. Such alternative 

30 storage devices should be considered equivalents. 
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1 The present invention, as described in certain embodiments herein, is 

2 implemented using a programmed processor executing programming instructions 

3 that are broadly described above in flow chart form that can be stored on any 

4 suitable electronic storage medium or transmitted over any suitable electronic 

5 communication medium. However, those skilled in the art will appreciate, upon 

6 consideration of this teaching, that the processes described above can be 

7 implemented in any number of variations and in many suitable programming 

8 languages without departing from the present invention. For example, the order of 

9 certain operations carried out can often be varied, additional operations can be 

10 added or operations can be deleted without departing from the invention. Error 

11 trapping can be added and/or enhanced and variations can be made in user 

1 2 interface and information presentation without departing from the present invention. 

1 3 Such variations are contemplated and considered equivalent. 

14 While the invention has been described in conjunction with specific 

1 5 embodiments, it is evident that many alternatives, modifications, permutations and 

16 variations will become apparent to those skilled in the art in light of the foregoing 

1 7 description. Accordingly, it is intended that the present invention embrace all such 

1 8 alternatives, modifications and variations as fall within the scope of the appended 

19 claims. 

20 What is claimed is: 
21 

22 
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